Text classification - Final.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5acc9c59",
   "metadata": {},
   "source": [
    "# <font color='red'>-----------------------------------------------------------------------------------------------------</font>\n",
    "# <font color='green'>Sentiment Labelled Sentences Data Set  - Text classification</font>\n",
    "## <font color='green'>Text Classification With Python and Keras (different model implementations)</font>\n",
    "# <font color='red'>-----------------------------------------------------------------------------------------------------</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24ff04c5",
   "metadata": {},
   "source": [
    "# <ins><div class=\"alert alert-block alert-info\">*Objective: -*</div></ins>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9395f76",
   "metadata": {},
   "source": [
    "### Customer Sentiment analysis from text reviews with the help of NLP (NLTK and Word2vec and modelling using Convolution neural network"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30372336",
   "metadata": {},
   "source": [
    "## <ins><div class=\"alert alert-block alert-info\">*Part 1: Extracting the dataset*</div></ins>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "091fdd0d",
   "metadata": {},
   "source": [
    "### <ins><div class=\"alert alert-block alert-warning\">*Step 1: Reading the text file to create a dataframe*</div></ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90055675",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "09fef5da",
   "metadata": {},
   "outputs": [],
   "source": [
    "# reading the files from the data source and creating dataframe with sentence, label, source\n",
    "text_dict = {'yelp':r'C:\\Users\\Indranil\\sentiment labelled sentences\\sentiment labelled sentences\\yelp_labelled.txt',\n",
    "            'imdb':r'C:\\Users\\Indranil\\sentiment labelled sentences\\sentiment labelled sentences\\imdb_labelled.txt',\n",
    "            'amazon':r'C:\\Users\\Indranil\\sentiment labelled sentences\\sentiment labelled sentences\\amazon_cells_labelled.txt'}\n",
    "\n",
    "reviews_list = []\n",
    "for source, filepath in text_dict.items():\n",
    "    reviews = pd.read_csv(filepath, names=['sentence', 'liked'], sep='\\t')\n",
    "    reviews['source'] = source  # Adding column for the source name\n",
    "    reviews_list.append(reviews)\n",
    "\n",
    "reviews = pd.concat(reviews_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a915ba6a",
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce027be2",
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "055ce31e",
   "metadata": {},
   "source": [
    "Our dataframe is built with the statements written as reviews, the ratings 'liked' and the source or the platform where the review was provided"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8edd6fee",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(4,3))\n",
    "sns.countplot(reviews['liked'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cff7cb62",
   "metadata": {},
   "source": [
    "#### We seem to have a pretty balanced data set in terms of -ve and +ve sentiments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "beb9d6fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# resetting index since dataframe is made with 3 sources and index is getting reset after end of items of the previous source\n",
    "reviews.reset_index(drop=True,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c98f01f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# dropping specific rows since during the pre processing those rows got converted to empty string\n",
    "reviews.drop(reviews.index[[140,1064,1590]],inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81f4bb79",
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dd77f4e",
   "metadata": {},
   "source": [
    "### <ins><div class=\"alert alert-block alert-warning\">*Step 2:Text pre-processing*</div></ins>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cee917a2",
   "metadata": {},
   "source": [
    "Text preprocessing steps to remove unwanted elemenmts like emails, special char etc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "075140d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing libraries for text pre processing\n",
    "import re \n",
    "import nltk\n",
    "\n",
    "nltk.download('stopwords')\n",
    "nltk.download('wordnet')\n",
    "\n",
    "from nltk.corpus import stopwords\n",
    "from nltk.stem import WordNetLemmatizer\n",
    "wrd_lemma = WordNetLemmatizer()\n",
    "\n",
    "all_stpwrds = stopwords.words('english')\n",
    "all_stpwrds.remove('not')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e84fb215",
   "metadata": {},
   "outputs": [],
   "source": [
    "#cleaning the data\n",
    "corpus = []\n",
    "for i in reviews['sentence']:\n",
    "      \n",
    "    cleaned = re.sub('[^a-zA-Z]',' ',i)\n",
    "    cleaned = cleaned.lower()\n",
    "    cleaned = cleaned.split()\n",
    "    cleaned = [wrd_lemma.lemmatize(word) for word in cleaned if not word in set(all_stpwrds)]\n",
    "    cleaned = ' '.join(cleaned)\n",
    "    corpus.append(cleaned) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5e2c27a",
   "metadata": {},
   "source": [
    "Looks like we have a cleaned dataset with preprocessed texts of the reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f2326eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# corpus"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e420aad6",
   "metadata": {},
   "source": [
    "### <ins><div class=\"alert alert-block alert-warning\">*Step 3: Processing the corpus for model building and word embedding*</div></ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45e72ba6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# defining X and y dataset\n",
    "X = corpus\n",
    "y = reviews['liked']\n",
    "\n",
    "# splitting the data into test and training data\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.30,random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02b4eed9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing libraries for deep learning models\n",
    "import tensorflow as tf\n",
    "import tensorflow.keras as keras\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4934be14",
   "metadata": {},
   "outputs": [],
   "source": [
    "# # creating a tokenizer\n",
    "tokenizer = Tokenizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d684af2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# building the word index\n",
    "tokenizer.fit_on_texts(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a38a228a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# converting strings into list of integer indices\n",
    "X_train_seq = tokenizer.texts_to_sequences(X_train)\n",
    "X_test_seq = tokenizer.texts_to_sequences(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62330c3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# displaying word vocabulary\n",
    "word_vocab = len(tokenizer.word_index)+1\n",
    "# print(tokenizer.word_index)\n",
    "# print(\"Found %s unique words\" % word_vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58de9a6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# # padding the sequences for having equal length of input to the model\n",
    "\n",
    "# finding the largest review sentence\n",
    "max_len=[]\n",
    "for i in X_train:\n",
    "    i = i.split()\n",
    "    max_len.append(len(i))\n",
    "\n",
    "max_length = max(max_len)\n",
    "print(max_length)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da8aacc6",
   "metadata": {},
   "source": [
    "The largest sequence in the list is 684. We shall use that as max length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45be0797",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_pad = pad_sequences(X_train_seq,maxlen=max_length,padding='post')\n",
    "X_test_pad = pad_sequences(X_test_seq,maxlen=max_length,padding='post')\n",
    "y_train = np.asarray(y_train)\n",
    "y_test = np.asarray(y_test)\n",
    "# print(\"Shape of the pad_data Tensor\",X_test_pad.shape)\n",
    "# print(\"Shape of the label Tensor\",y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dfe51a96",
   "metadata": {},
   "source": [
    "### <ins><div class=\"alert alert-block alert-warning\">*Step 4:Data transformation: Transform the cleaned dataset into vector of words using word embeddings*</div></ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f9e17b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# loading word2vec word embeddings\n",
    "import gensim\n",
    "wordembeddings = gensim.models.KeyedVectors.load_word2vec_format('D:/GoogleNews-vectors-negative300.bin',binary=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "407532b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "tot_words = word_vocab\n",
    "embed_dim = 300\n",
    "embed_matrix = np.zeros((tot_words,embed_dim))\n",
    "skipped_words = []\n",
    "embed_vect = np.empty(shape=(300,))\n",
    "\n",
    "for words,index in tokenizer.word_index.items():\n",
    "    ######## word embedings dont have OOV word and hence will throw error\n",
    "    if words in wordembeddings:\n",
    "        embed_vect = wordembeddings[words]\n",
    "    else:\n",
    "        skipped_words.append(words)\n",
    "        \n",
    "    embed_matrix[index] = embed_vect        "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5fe0291",
   "metadata": {},
   "source": [
    "### <ins><div class=\"alert alert-block alert-warning\">*Step 5:Model building: Fitting Deep learning model, model testing*</div></ins>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f37d1605",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create the embedding layer\n",
    "from tensorflow.keras.models import *\n",
    "from tensorflow.keras.layers import *\n",
    "embed_layr = Embedding(tot_words,embed_dim,weights=[embed_matrix],input_length=max_length,trainable=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1719da8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# building the model\n",
    "model = Sequential()\n",
    "\n",
    "# building the convolution layers\n",
    "model.add(embed_layr)\n",
    "model.add(Conv1D(filters=64, kernel_size=2, activation='relu'))\n",
    "model.add(MaxPooling1D(pool_size=2))\n",
    "model.add(Conv1D(filters=128, kernel_size=2, activation='relu'))\n",
    "model.add(MaxPooling1D(pool_size=2))\n",
    "model.add(Conv1D(filters=256, kernel_size=2, activation='relu'))\n",
    "model.add(MaxPooling1D(pool_size=2))\n",
    "model.add(Conv1D(filters=512, kernel_size=2, activation='relu'))\n",
    "model.add(MaxPooling1D(pool_size=2))\n",
    "model.add(Flatten())\n",
    "\n",
    "# building the dense layers\n",
    "model.add(Dense(64, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(32, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(10, activation='relu'))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "print(model.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92ace306",
   "metadata": {},
   "source": [
    "**Training the model**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5eaedef0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.optimizers import *\n",
    "\n",
    "# compile network\n",
    "model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
    "# fit network\n",
    "model.fit(X_train_pad, y_train, batch_size=64,epochs=9, verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38a075b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# evaluate\n",
    "loss, acc = model.evaluate(X_test_pad, y_test, verbose=2)\n",
    "print('Test Accuracy: %f' % (acc*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca30e252",
   "metadata": {},
   "source": [
    "**Training accuracy is at ~90% plus and Test accuracy is at ~82% plus. Our model has worked appropriately**<br>\n",
    "**Let's test our model by giving some reviews as example**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3300d78",
   "metadata": {},
   "outputs": [],
   "source": [
    "# inp = str(input(\"Enter review to find sentiment: \"))\n",
    "# a = [inp]\n",
    "# a_seq = tokenizer.texts_to_sequences(a)\n",
    "# a_pad = pad_sequences(a_seq,maxlen=max_length,padding='post')\n",
    "# predict = model.predict(a_pad)\n",
    "# print(predict)\n",
    "# if predict > 0.5:\n",
    "#     print(a,'\\n')\n",
    "#     print(\"Positive sentiment\")\n",
    "# else:\n",
    "#     print('Negative sentiment')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "843d72e3",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}